Skip to content

fix(server): paginate large thread history to stop the server running out of memory - #3510

Closed
olafura wants to merge 13 commits into
pingdotgg:mainfrom
olafura:fix/sqlite-activity-pagination
Closed

fix(server): paginate large thread history to stop the server running out of memory#3510
olafura wants to merge 13 commits into
pingdotgg:mainfrom
olafura:fix/sqlite-activity-pagination

Conversation

@olafura

@olafura olafura commented Jun 22, 2026

Copy link
Copy Markdown

Fixes the server-side root cause behind #2761 (and likely #996).

What Changed

The server used to load a thread's entire history — every tool activity, message, and checkpoint — into memory at once. On a busy database that's hundreds of MB, which exhausted the Node heap and crashed the server (and made large threads slow to load / churn reconnects, as reported in #2761).

Now the server loads only the most recent slice of a thread's activity and tells the client whether older history exists; the web and mobile apps fetch older pages on demand as you scroll up. Deep history is still fully reachable — it's paged in instead of loaded all at once.

  • Server: the snapshot endpoint and the t3 CLI offline path stop loading every thread's full history (they only read the project list). Thread detail is bounded to the most recent activities, with a hasMoreActivities flag and a cursor-paginated getThreadActivities RPC for older pages. The older-page query is shaped to use the existing index instead of a full sort.
  • Reconnect catch-up: shell/thread subscriptions replay only a recent gap (up to 1,000 global events); older cached cursors receive a fresh projection snapshot instead of traversing the full event log.
  • Web / Mobile: lazy-load older history on scroll-up — prepend older pages, anchored so the viewport doesn't jump.

The commits remain separated by server, web, mobile, shared-client, reconnect-safety, and rebase-formatting concerns so each layer is independently reviewable.

Why

Programs in a thread emit a lot of tool activity over time. The server treated "open a thread" (and "load the snapshot") as "read all of it from SQLite into memory." For a long-lived thread or a busy database, that single read materialised more than the heap could hold, so the process ran out of memory and died — and even when it didn't crash, shipping one giant snapshot blocked the connection (#2761). Bounding the read + paging older history on demand keeps memory flat regardless of how much history a thread accumulates.

UI Changes

The web and mobile timelines gain a small "Load older history" affordance and a loading spinner when you scroll to the top of a thread longer than the window. It's a minor, additive interaction. I wasn't able to attach before/after screenshots or a scroll video here — happy to add them, or the server commit alone (no UI) carries the actual reliability fix if you'd rather review that first.

Checklist

  • I explained what changed and why
  • before/after screenshots for UI changes — to be added (web/mobile lazy-load affordance)
  • video for the scroll-up interaction — to be added
  • The server fix is independently reviewable (first commit, no UI)

🤖 Generated with Claude Code


Note

High Risk
Changes core read paths (bounded thread detail, snapshot/CLI switching to command read model, subscribe catch-up behavior) and pagination edge cases for sequenced vs legacy activities; any consumer expecting a full snapshot or unbounded replay could break.

Overview
Server memory and API: Thread detail now loads only the 500 most recent activities and sets hasMoreActivities when more exist. getThreadActivities pages older rows via sequence or legacy (createdAt, activityId) cursors. The orchestration HTTP snapshot and t3 CLI offline path use getCommandReadModel instead of materializing full per-thread tables. subscribeThread replays at most 1,000 global events after the cursor; otherwise it emits a replacement thread snapshot (with live events buffered so nothing is lost).

Clients: New useOlderThreadActivities in client-runtime prepends paged history, dedupes, and resets on live-window reshape. Web and mobile scroll-up (and explicit controls) load older pages; the feed and composer use merged activities so approvals/user-input prompts still surface after scrolling back. Mobile picks the newest open pending request when multiple exist in the merged set. Sidebar thread prewarming is removed.

Contracts: OrchestrationThread gains optional hasMoreActivities; HTTP snapshot success type documents empty thread payloads on the command read model.

Reviewed by Cursor Bugbot for commit eac66d8. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Paginate large thread activity history to prevent server memory exhaustion

  • Caps thread detail snapshots to the 500 most recent activities and adds a hasMoreActivities flag on OrchestrationThread payloads when older history exists.
  • Adds a new orchestration.getThreadActivities WebSocket RPC to fetch older activities using a sequence-based or legacy unsequenced (createdAt/id) cursor, with a max page size of 500.
  • Bounds subscribeThread catch-up replay to 1000 events; for stale or invalid cursors, sends a thread snapshot followed by buffered live events instead of attempting unbounded replay.
  • Adds useOlderThreadActivities hook in client-runtime and wires it into ChatView (web) and useThreadComposerState (mobile) to support infinite scroll-up with deduplication and reshape detection.
  • The MessagesTimeline (web) and ThreadFeed (mobile) components now render a 'Load older history' button or loading indicator at the top, with auto-load on scroll to start.
  • Risk: the orchestration/snapshot HTTP endpoint and offline CLI snapshot now return the command read model (getCommandReadModel) instead of the full snapshot, which is a breaking change for any consumers relying on the full activity history from that endpoint.

Macroscope summarized eac66d8.

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 34ea15ec-9247-45dc-9bbe-39429d23002c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size:XL 500-999 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list. labels Jun 22, 2026
Comment thread apps/web/src/components/ChatView.tsx Outdated
Comment thread apps/web/src/components/ChatView.tsx Outdated
Comment thread apps/web/src/components/ChatView.tsx Outdated
Comment thread apps/web/src/components/ChatView.tsx Outdated
olafura added a commit to olafura/t3code that referenced this pull request Jun 22, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@olafura

olafura commented Jun 22, 2026

Copy link
Copy Markdown
Author

Thanks Bugbot — addressed in cddc7e1:

  • Reconnect leaves stale older pages / Revert keeps lazy-loaded activities (both): the lazy-load state now resets when the live window's oldest activity id changes, not just on thread switch. That id is stable while activities only append, so it doesn't fire during a normal turn — but a reconnect re-snapshot or a checkpoint revert that drops live rows changes it, invalidating the prepended older pages. Applied to web and mobile.
  • Web skips live activity dedup: web now dedups a new older page against both the already-loaded older pages and the live window (mobile already did this), so a boundary overlap can't produce duplicate ids.
  • Unsequenced cursor hides sequenced history: I believe this is a false positive. Legacy NULL-sequence rows always sort as the oldest in both the detail window (sequence DESC, NULLs last) and the pager. So the oldest-loaded row becomes unsequenced only after every sequenced row is already in the loaded set — there are never sequenced rows older than an unsequenced one for the sequence IS NULL cursor to strand. Happy to add a test if useful.

Comment thread apps/web/src/components/ChatView.tsx Outdated
Comment thread apps/mobile/src/state/use-thread-composer-state.ts Outdated
Comment thread apps/web/src/components/ChatView.tsx Outdated
@macroscopeapp

macroscopeapp Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

Introduces a new RPC endpoint and pagination infrastructure across server and clients, fundamentally changing how thread activities are loaded (windowed to 500 with lazy-load). While addressing memory concerns, this is substantial new capability with cross-cutting runtime behavior changes rather than a simple bug fix.

You can customize Macroscope's approvability policy. Learn more.

Comment thread apps/web/src/components/ChatView.tsx Outdated
olafura added a commit to olafura/t3code that referenced this pull request Jun 22, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from cddc7e1 to 75d25e2 Compare June 22, 2026 20:48
@olafura

olafura commented Jun 22, 2026

Copy link
Copy Markdown
Author

Addressed the latest review in 75d25e266 (rebased on current main):

  • Stale in-flight load after reset — the reset now bumps a generation counter (olderActivitiesGenRef); the in-flight load captures it at dispatch and drops its result in .then()/.finally() if it changed, so a late page can't repopulate the cleared state. Replaces the thread-key-only staleness check (which was unchanged on a same-thread reshape).
  • Revert may skip reset / Reconnect leaves stale pages — the reset now also fires when the live window shrinks (a checkpoint revert that drops rows without changing the oldest row), not just when the oldest id changes (reconnect). A pure append (same oldest, larger count) keeps the loaded pages. Applied to web and mobile.
  • Empty page churns / loops — when dedup leaves nothing new, the handler returns without creating a new array reference and stops, so an unadvanced cursor can't loop identical requests.
  • Web skips live dedup — dedups against the full merged set (older + live).
  • Unsequenced cursor hides sequenced history — I believe this is a non-issue and added a server regression test to prove it: NULL-sequence rows always sort oldest, so when the oldest loaded row is unsequenced every sequenced row is already in the window; the sequence IS NULL cursor reaches all 103 older unsequenced rows in the test without stranding any of the sequenced ones.

Server/web/mobile typecheck, lint, and the projection + timeline suites pass.

Comment thread apps/web/src/components/ChatView.tsx Outdated
Comment thread apps/web/src/components/ChatView.tsx Outdated
olafura added a commit to olafura/t3code that referenced this pull request Jun 22, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from 75d25e2 to 618df70 Compare June 22, 2026 21:14
@olafura

olafura commented Jun 22, 2026

Copy link
Copy Markdown
Author

Addressed the latest review round:

Cursor — "Append clears lazy-loaded history" (Medium) — The reset effect used liveThreadActivities[0]?.id as the live window's oldest boundary. The reducer's activityOrder sorts unsequenced rows to the end (a missing sequence is treated as newest) while the server snapshot lists legacy unsequenced rows first, so the first live append re-sorted the array and shifted index 0 — making a plain append look like a window reshape and wrongly clearing the user's scrolled-up history. Replaced it with an order-independent oldest sentinel, liveWindowOldestActivityId (min by createdAt, then id), which is stable across appends (always newest) and only changes on a genuine reshape (reconnect re-snapshot / revert). Applied to both ChatView and the mobile composer; the helper lives in client-runtime with unit tests.

macroscope — mixed-thread flash (Low, ChatView.tsx:1724) — Moved the older-pages reset from useEffect to useLayoutEffect so the cleared state commits before paint; otherwise a thread switch rendered one frame with the previous thread's lazy-loaded pages still merged in, flashing stale work-log/approval rows. Mirrored in mobile.

The two earlier macroscope Mediums (1703 staleness guard, 1762/1786 empty-fresh loop) are already ✅ resolved in the prior commit — the generation-ref guard supersedes the request-key check, and the empty-fresh path early-returns without churning the array. The High 1772 ("unsequenced cursor hides sequenced history") remains a false positive: the lazy-load cursor (threadActivities[0]) is only unsequenced when the whole thread fits the window, in which case hasMoreActivities is false and no sequenced rows are stranded.

pnpm --filter @t3tools/web typecheck, --filter @t3tools/mobile typecheck, the client-runtime reducer suite (24 passing incl. the new helper specs), and lint are green. Squashed into the lazy-load-hardening commit and rebased on upstream/main.

Comment thread packages/client-runtime/src/state/threadReducer.ts
Comment thread apps/web/src/components/ChatView.tsx Outdated
@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from 618df70 to c8e6474 Compare June 22, 2026 22:02
@olafura

olafura commented Jun 22, 2026

Copy link
Copy Markdown
Author

Latest round:

Cursor — "Wrong pagination cursor after live sort" (High). Valid consistency gap I introduced last round. I'd made the reshape sentinel order-independent (liveWindowOldestActivityId = chronological-oldest) but left the pagination cursor on threadActivities[0], which the reducer can fill with a newer sequenced row (it sorts unsequenced rows to the end). Those two now agree: extracted oldestActivityByChronology(activities) and made the cursor page from it in both ChatView and the mobile composer, so paging always continues from the genuinely-oldest loaded row (driving the unsequenced beforeCreatedAt/beforeActivityId cursor when that row is legacy-unsequenced, instead of a beforeSequence that could skip it). The sentinel now delegates to the same helper. Added a reducer test asserting the cursor-oldest is the unsequenced legacy row and that the sentinel agrees.

Cursor — "Stale hasMore after revert" (Medium). False positive. thread.reverted keeps activity.turnId === null || retainedTurnIds.has(activity.turnId) — it drops activities for the reverted (newer) turns, never the older history. hasMoreActivities describes whether older rows exist beyond the window, which a revert of newer turns doesn't change, so retaining the flag is correct. (And even if it were stale-true, the lazy-load already no-ops on an empty page: fresh.length === 0 clears hasMore without looping.)

@t3tools/web + @t3tools/mobile + @t3tools/client-runtime typecheck clean, reducer suite 26 passing, lint clean. Squashed into the lazy-load-hardening commit and rebased on upstream/main.

Comment thread apps/web/src/components/ChatView.tsx Outdated
Comment thread apps/web/src/components/ChatView.tsx Outdated
@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from c8e6474 to 848cff6 Compare June 22, 2026 22:24
@olafura

olafura commented Jun 22, 2026

Copy link
Copy Markdown
Author

Self-review (10-angle, max effort) of 848cff64

Ran an adversarial multi-angle review and grounded the findings against the real 1.5 GB database (131,532 activities). Posting the results.

Key empirical finding

The projection_thread_activities.sequence column is 100% NULL in every database I can observe (both the 4 MB dev DB and the 1.5 GB realistic one), and 0 threads have a NULL-sequence row newer than a sequenced one. runtimeEventToActivities only sets sequence from event.sessionSequence, which no provider currently emits. So the beforeSequence/sequenced-row pathway is never exercised in production — all paging goes through the created_at keyset. This reframes the scary-looking findings as latent, not live.

Fixed

Sev Finding Fix
Low (hardening) The internal SQL-input schemas typed beforeSequence/limit as bare Schema.Number, looser than the contract's NonNegativeInt. The WHERE clause (sequence < beforeSequence OR sequence IS NULL) is only equivalent to the old COALESCE(sequence,-1) < beforeSequence for non-negative cursors — a negative one silently returns only unsequenced rows. Tightened both internal schemas to NonNegativeInt so any future non-RPC caller is validated, not just the RPC boundary. Typecheck + 11 pager tests green.

Latent / by-design (NOT changed — flagging for a decision)

  • Window vs reducer NULL-ordering disagreement (highest-severity-if-triggered): the server window orders sequence DESC (SQLite → NULLs last/oldest) while the client reducer's activityOrder uses sequence ?? MAX_SAFE_INTEGER (NULLs newest). They only agree today because every row is NULL. If sequence is ever partially backfilled, a thread with mixed rows would (a) drop the newest unsequenced rows from the 500-window and (b) paginate in a different order than it displays. The robust fix is to order both sides by created_at consistently — but that's an index-strategy change (the covering index leads with sequence) and a no-op today, so I've left it. Recommend either ordering by created_at end-to-end, or guaranteeing sequence is all-or-nothing.
  • Speculative sequence machinery: given the above, the entire sequenced-cursor arm (second SQL query, union input arm, "beforeSequence" in input branch, the sequence-leading index) is dead in practice. A single (created_at, activity_id) keyset would cover 100% of today's reality with ~half the surface. Kept as forward-looking, but worth a conscious call.
  • web/mobile duplication (AGENTS.md "Duplicate logic across multiple files … should be avoided"): the ~120-line lazy-load state machine (gen guard, in-flight ref, reshape useLayoutEffect, dedup) is near-verbatim in ChatView.tsx and use-thread-composer-state.ts. Should become one shared hook — deferred because it's a cross-app React-architecture refactor, not a surgical fix.
  • Asymmetry: activities are windowed to 500 but messages in the same getThreadDetailById are still unbounded, so the heap-OOM rationale is only half-applied.
  • handleScroll/loadOlder identity churn on every live append (perf, not correctness); THREAD_DETAIL_ACTIVITY_WINDOW overloaded as window size + default page + max page.

Verified safe (no action)

getSnapshotgetCommandReadModel (only the CLI reads .projects); toReversed() (Node/lib supports it); mapThreadActivityRow typing; hasMoreActivities optional doesn't break fixtures/decoders; window slice/hasMore arithmetic; the empty-page stop guard; sequence===0 cursor; created_at tie-break cursor. No CLAUDE.md/AGENTS.md violations beyond the duplication note.

@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from 848cff6 to ead248b Compare June 22, 2026 22:31
@olafura

olafura commented Jun 22, 2026

Copy link
Copy Markdown
Author

Addressed the latest review round (ead248b4)

Three threads — two fixed, one verified as a false positive. Both fixes applied to web (ChatView) and mobile (use-thread-composer-state) identically.

🟡 Cursor — "Duplicate page ends pagination early" (r3455736449) — Fixed. Paging now runs off an explicit olderCursorRef that advances to each page's oldest row (pages are ascending, so [0] is oldest) on every successful page — even one that dedupes to zero new rows. Because the server cursor is strict, the cursor strictly decreases, so an all-overlap page keeps paging (respecting the server's page.hasMore) without re-requesting the same cursor or looping. The old setOlderHasMore(false) early-stop is removed.

🟢 macroscope — "stale dedup seen" (r3455790594) — Fixed as suggested. Added a threadActivitiesRef (mobile: mergedActivitiesRef) assigned the merged array every render; the .then() dedup now builds seen from …Ref.current instead of the closure-captured snapshot, so a live append or a prior prepend that settles mid-flight is always reflected.

🟡 Cursor — "Stale hasMore after revert" (r3455516138) — False positive. thread.reverted keeps turnId === null || retainedTurnIds.has(turnId), dropping activities for the reverted (newer) turns and keeping older ones. hasMoreActivities describes whether rows exist older than the window, which a newer-turn revert doesn't change — so retaining it is correct. No code change.

These were latent (reachable only in the mixed sequenced+unsequenced case, which never occurs — sequence is 100% NULL in the real DB), but the fixes make paging correct-by-construction rather than relying on that invariant.

Verified: @t3tools/web + @t3tools/mobile typecheck clean, lint clean, reducer (26) / MessagesTimeline (12) / pager (11) suites green. Squashed into the lazy-load commit and rebased on upstream/main.

olafura added a commit to olafura/t3code that referenced this pull request Jun 24, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from ead248b to 311b469 Compare June 24, 2026 10:11
olafura added a commit to olafura/t3code that referenced this pull request Jun 24, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
patroza added a commit to patroza/t3code that referenced this pull request Jul 25, 2026
…#3510)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
patroza added a commit to patroza/t3code that referenced this pull request Jul 25, 2026
…#3510)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
patroza added a commit to patroza/t3code that referenced this pull request Jul 25, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
olafura added a commit to olafura/t3code that referenced this pull request Jul 27, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
olafura added a commit to olafura/t3code that referenced this pull request Jul 27, 2026
…fixes

- OSC 10/11/12 rgb in output is the legitimate set-colour command — kept
  (stripFromOutput flip in the sequence grammar), reply form still stripped
  from input.
- sanitizePersistedTerminalHistory bypasses overflow recovery: whole-buffer
  loads preserve an over-cap unterminated tail byte-for-byte.
- CPR input strip is query-gated (lastCursorQueryRelayedAt + 5s grace): the
  byte-identical modified-F3 keystroke passes at a quiet prompt while the
  querying-prompt flood stays broken.
- Mobile surfaces the NEWEST open approval/user-input request instead of the
  oldest, so lazy-loaded ancient requests can't hijack the current prompt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
olafura added a commit to olafura/t3code that referenced this pull request Jul 27, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from 034f493 to 8494b3e Compare July 27, 2026 10:20
Comment thread apps/server/src/ws.ts Outdated
Comment thread apps/server/src/ws.ts
patroza added a commit to patroza/t3code that referenced this pull request Jul 27, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
patroza added a commit to patroza/t3code that referenced this pull request Jul 27, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
patroza added a commit to patroza/t3code that referenced this pull request Jul 28, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
olafura added a commit to olafura/t3code that referenced this pull request Jul 28, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
olafura added a commit to olafura/t3code that referenced this pull request Jul 28, 2026
…fixes

- OSC 10/11/12 rgb in output is the legitimate set-colour command — kept
  (stripFromOutput flip in the sequence grammar), reply form still stripped
  from input.
- sanitizePersistedTerminalHistory bypasses overflow recovery: whole-buffer
  loads preserve an over-cap unterminated tail byte-for-byte.
- CPR input strip is query-gated (lastCursorQueryRelayedAt + 5s grace): the
  byte-identical modified-F3 keystroke passes at a quiet prompt while the
  querying-prompt flood stays broken.
- Mobile surfaces the NEWEST open approval/user-input request instead of the
  oldest, so lazy-loaded ancient requests can't hijack the current prompt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
patroza added a commit to patroza/t3code that referenced this pull request Jul 29, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
patroza added a commit to patroza/t3code that referenced this pull request Jul 29, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
@olafura
olafura force-pushed the fix/sqlite-activity-pagination branch from 8494b3e to d7ddba7 Compare July 29, 2026 12:55

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d7ddba7. Configure here.

Comment thread apps/server/src/ws.ts Outdated
olafura added a commit to olafura/t3code that referenced this pull request Jul 29, 2026
Address review (PR pingdotgg#3510, Cursor Bugbot):

- Reconnect / checkpoint revert can re-snapshot or filter the live activity
  window, but the prepended `olderActivities` weren't invalidated — leaving gaps
  or showing reverted history. Reset the lazy-load state when the live window's
  oldest activity id changes (it's stable while activities only append, so this
  doesn't fire during a normal turn), in addition to on thread switch. Web + mobile.
- Web only deduped a new older page against already-loaded older pages, not the
  live window (mobile already did both). Dedup against both so a boundary overlap
  can't produce duplicate ids / React keys.

Not changed — the "unsequenced cursor hides sequenced history" finding is a false
positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor
ordering, so the oldest-loaded row is only unsequenced once every sequenced row
is already loaded; the unsequenced cursor can never strand sequenced rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
olafura added a commit to olafura/t3code that referenced this pull request Jul 29, 2026
…fixes

- OSC 10/11/12 rgb in output is the legitimate set-colour command — kept
  (stripFromOutput flip in the sequence grammar), reply form still stripped
  from input.
- sanitizePersistedTerminalHistory bypasses overflow recovery: whole-buffer
  loads preserve an over-cap unterminated tail byte-for-byte.
- CPR input strip is query-gated (lastCursorQueryRelayedAt + 5s grace): the
  byte-identical modified-F3 keystroke passes at a quiet prompt while the
  querying-prompt flood stays broken.
- Mobile surfaces the NEWEST open approval/user-input request instead of the
  oldest, so lazy-loaded ancient requests can't hijack the current prompt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
patroza added a commit to patroza/t3code that referenced this pull request Jul 29, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
patroza added a commit to patroza/t3code that referenced this pull request Jul 29, 2026
…#3510) (#35)

Source: pingdotgg#3510
Source SHA: 034f493

Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks.
Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes.
Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants